A. 



4. Types of objectives (motor performance, verbal, 
and discrimination) 

5. Taxonomy of objectives (cognitive* affective > 
and psychomotor domains) . , 

IMPLEMENTATION NEEDS 

professional person with a broad knowledge of and practical 
experience in the area of behavioral ob jdc^ves would be needed 
:o present this program and, iw>re aspeciall^to answer ques~ 
tions related to it. Also needed would be an airhead projector. 
The presentation could be made in approximately pWour's time. 

SPECIAL FEATURES ■ > \ 

* 

The content of this program is broacl-based, encompassing a wider, 
range of material thaln many other models. 



s 
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Instructional Objectives 



/ Developers : 

> Caroline Metheny Dillnjan 
Harold F#3 Rahmlow 

Source:' # ) , 

Fearon Publishers 
# Lear Sigler, Inc., 
Education Division 
jS David Drive m 
Belmont, Ca. 94002 

Cost: 
\. $2. 50/baok 

Copyright: 
Yes^ 1972 




OVERVIEW:. 



JC 



Written primarily for teachers , the main thrust of this book, 
as the title indicates ,1 is the writing of instructional objec- 
tives. The authors 9 goals are as follows : 

1. To acquaint the reader with resource materials on 
objectives and suggest ways they might be used. 

2. To list and explain useful points to be considered 
— — when-wri ting ob j ecti ves . 

3. To explain the importance of and a technique for 
writing test items for objectives. 

4. To present varying viewpoints on the use of verb 
forms most commonly used for objectives. 

5. To discuss style technicalities. 

6. To illustrate steps in how an objective is developed 
from rough draft form to an acceptable finished 
product. 

7. To give some original objectives and provide 
opportunity for revision and evaluation. 

PROCEDURE : 

This model was designed for use by the individual. Although 
suggested references are given, this book provides very little 
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^ background reading , as the authors presume the reader has al- 
ready done, research previous to his encounter with this par* 
ticular program* 

The following is the checklist for objectives suggested by 
Dillman and Rahmlows 

1. Level of specificity: Is the objective too specific 
or too broad? 

' 2. Principal performance: Is the skill called for in the 

objective the principal performance stmght , or is it 
an indicator behavior? 

3. Overt behavior: * Is the behavior observable and can 
it be evaluated? 

4. Method or process: Has the method, if it is important, 
been specified? 

5t Evaluation or performance criteria: Are criteria for 
evaluation included in the objective? 

6. Relevant conditions: Have any relevant conditions, 
been included? 

7. Student-directed performances Is the objective 
directed to the student? / 

8. Appropriate reading level and vocabulary: Is the ob- 
jective written in appropriate vocabulary at ah ap- 
propriate reading level? 

Also included is a sample checklist for test items. 

* i - 

Using the suggested worksheet, this is the step-by-step pro- 
cedure for the actual writing of the objectives: 

1. Jot down the general idea for an objective. 

2. Write on the worksheet. a, first rough draft. 

3. Write a sample test item. 

4. Use the checklist to review the objective. 

5. Review the sample test item. 

6. Copy the corrected second draft in the spacfe provided 
on the worksheet. 

7. Read over the final version, making any necessary 
editorial and grammatical changes. 

Twenty- five sample objectives are given for revision by the 
r reader. These samples deal mainly with subject content 

(cognitive), rather than the affective or psychomotor domain, 
and range in level from first gride through adult education. 

— — es 



r 



IMPLEMENTATION NEEDS: - 

Extra duplicated copies of the sample worksheet might be help- 
ful in conducting this program. A coordinator or leader might 
be useful in helping the participants evaluate their perfor- 
mance in the practice exercises* 

SPECIAL FEATURES : 

An element of humor is used to emphasize some important points* 
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Writing Performance Objectives 

A Programmed Course for the Writing of Performance Objectives 
Instructor's Manual for Teachers and Administrators 



Developer: 

Program Development Center 
of Northern California 
*Butte County Schools Office 
Chico, Ca. 95926 

f 

Source: 

Commission on Educational 
" Planning 

-Phi Delta Kappa, Inc. 
P.O. Box 789 
Bloomington, In. 47401 

"Cost z 

Model t is Part II of PDK 
Goal Setting Kit $3.00. 
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Copyright: 
No (1972) 



OVERVIEW:- 



This twq-manual program, extensxve 
and somewhat technical, is written 
primarily for classroom teachers 
and their instructor. One booklet 
contains programmed material for 
participants and the other contains 
background material/ lesson plans for group meeting, worksheets, 
and other essential information for the instructor. 



Upon completion of the program, 
to db the following: 



the participants will be able 



1. Identify 2 terms: goal statements and similar goals. 

2. Identify the 3 parts of a performance objective* 

3. Write 5 performance objectives. 

4. Write cognitive objectives. 

5. Recognize psychomotor objectives. ' 

6. Recognize and write affective objectives. 

7. Recognize and write 3 levels of performance 
objectives* 



PROCEDURE; 



This program, to be completed in four weeks, includes indepen* 
jient and group activities which are designed to improve one's' 
^skill in writing performance objectives. Between each of the 
four consecutive weekly meetings, the participants'are required 
to work independently through the programmed material contained 
in the student's booklet. Listed on the next -page are the 
procedures to be followed: 



Meeting 1 - Orientation meeting in which the instructor 
will outline procedures to be followed and /J 
to present the rationale for using performance 
objectives. / 
Independent Activity 1 - Complete Frames 1 through 34 of 
the Programmed Material. 
- — ^ Tteeting 2 - A critique of the objectives participants 

have written and group discussion and 
instruction. 

Independent Activity 2 - Complete Frames 35 through 115 

in the booklet. 
Meeting 3 - Critique of the objectives participants have 

written and, group discussion and instruction. ' 
Independent Activity 3 - Complete Frames 1^6 through 136. 
Meeting 4 - (Concluding meeting) Similar in format to 

the preceeding meetings. 

The pages following the programmed items include 6 assignments. 
They were designed to be completed during training sessions. 

IMPLEMENTATION NEEDS: 

A booklet is required for each participant in this gzogram, 
and a manual is needed for each instructor. A series of four 
consecutive weekly meetings , approximately an hour and a half 
to two hours in length, is required to execute this . course of 
study, as well ^s independent time between each meeting for 
the participants to work on the programmed material. Profes- 
sional personnel, proficient in the area of performance objec- 
tives,, is essential, to conduct the four structured meetings. 



SPEC I 'Mi 



?\Ti:RES 



Unlike many other models, this program is a blending of group 
and individual activities. Clear directions are given for . 
both types of activities. 
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Writing and Using Behavioral Objectives 

A Learning Packet for Teachers, Students, and Administrators 



Developers: v 
Sara Leles 
Raymond Bernabei 

Source: 

University Supply Store 
University of Alabama 

34586 

Cost: 1 
$3,50 

Copyright : 

Yes (Lem Associates) 
1972 
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OVERVIEW: * ' • 

Teachers (in- training and in-service), supervisors, curriculum 
directors, and administrators are the professionals for whom f 
this program was written. It was designed to aid participants 
in thinking about objectives and to grapple with ways in which 
objectives may be utilized for meaningful selections of content, 
methods, materials, and evaluation procedures. This packet was 
written to help the reader learn the following: 

\ 1. How to identify meaningful objectives. 
2.. How to write meaningful objectives. 
3; . How to use meaningful objectives. 

PROCEDURES : 

In order to facilitate the meeting of the objectives of this 
program, approximately 25 pages Of background .material are 
given. It includes information on the development of the use 
of behavioral objectives, the cognitive, affective, and psy- 
chomotor domains, and four basic characteristics of behavioral 
objectives. This information is followedXby numerous practice 
exercises designed to reinforce skill in identifying, writing, 
and using meaningful objectives. A pretest and ppsttest are 
included to determine the participant's gain. 
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The packet, culminates in a bu22 session which requires parti- 
cipants (organized into small groups of 5 to 7 members) to 
translate theory into practive and, more specifically, to write 
and to use objectives. A method for the selectipn of group 
leaders and recorders is outlined. 

. Following the buz2 session, all participants meet together ahd 
the. group recorders are asked to share the results of their 
group discussions. 

IMPLEMENTATION NEEDS ; ' , " 

The authors give no indication as to the 7 anticipated time re- 
quired to complete this packet* Although the time factor would 
vary according to how this booklet is used, several hours would 
be a Minimum estimate. Someone would ^e needed to organize the 
small groups for dispussion and to lead the concluding' session. 
r r in such a way that it is a meaningful experience for the pat- 
ticipants. - ^ 

"W ■ . ) . i 

SPECIAL FEATURES: . — - ■* 



This program, like few others, includes individual activities, 
as well as, small and large group activities. 
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Writing Worthwhile Behavioral Objectives. 



Developer : 
Julie S, Vargas 

Source: 

Harper/Row, Publishers Inc. 
49 East 33rd Street 
Hew York, N. Y. 10016 

Cost: 

Contact publisher. 

, i 

Copyright : 
1912* 
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OVERVIEW: 

This is a selfstinstructional book designed to help you write 
worthwhile, behaviorally stated teaching objectives that will 
increase the values of your courses and their relevance to Ypw* 
students* life outside school. It does not teach how to write 
objective! for attitudes (though the topic is considered brief-, 
ly)t it is restricted to what is often called the cognitive, 
domain: 

After completing the book the user should be capable of: 



1. 

2. 



3, 
4. 

5. 

6. 



selecting from a list of ten to twenty objectives the 
five that mre the most behavioral. 

telling which of the necessary criteria (referring to the 
behavior of the student, | describing observable behavior, 
and specifying a level of criterion of acceptable perfor- 
mance) a given objective ', meets qr fails, 
revising nonbehavioral objectives. 

writing behavioral objectives for general objectives or 
test items. 

classifying given objectives in the major categories of 
Bloom's taxonomy of educational objectives, 
replacing what Bloom calls "knowledge level objectives" 
with objectives that fall in the higher categories of 
Bloom's taxonomy. 
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7. given a textbook chapter, writing objectives for a unit 
using that chapter. 



The unit^should have all levels of Bloom's taxonomy represented. 



PROCEDURES: 

This book contains both text and exercises. The text presents 
the arguments for quality "objectives;* the exercises teach you 
how to write them* To avoid unnecessary practice, each dhapter 
of exercises begins with a pretest covering the skills tau|fite^ ? 
If you scpre 90 percent or letter on the pretest, you may s^h* 
the chapter and go on to th^ next. If you score less than 9( 
percent, then you are expected to do the exercises. And, of 
course, you can wo$k through the book at your own speed. 
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C.S.E. Hierarchial Objectives Charts 



Source: 
Cost: 

Copyright: 



Center for the Study of Evaluation 

University of California 

Los Angeles, California y 



$12.00 



NO 



^CONTENT And organization 

Goals and Objectives of elementary sch<5ol education are o|t- 
' £ined in chart form. Forty-one goal areas and one hundred and 
four sub-goals or objectives hav5 been categorized in a hier- 
archy. The charts contain objectives ranging from broad goal 
areas to more # specif ic objectives which lead to but do not in- 
clude behavioral objectives. /The goals cover both cognitive 
and affective areas and range from such categories as Develop- 
ment of Personality and Social Development to Geometry, Soci- 
ology, and Reasoning 

The format of the charts is best described by illustration. 
(See "Sample Objectives" on next page.) 



IPEL 



i :<fs> 



1 

Higher level thinking skills are covered in these charts as 
wsll as affective learnings. In addition, this hierarchical 
format is not coirimonly found among the o^her objectives banks 



/ 



The charts may be used in*the development \f a criterion-* 
referenced evaluation system. \ 
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."uM^te obj KCT iveg.: (Format of Che 




\ Parts of Speech 



• fcnowt masi..ng and various uhi of nount 

b knows mesnmg snd various uses of verbs md tenses snd conjugshon* 

c Knows meaning and various uses o* edteCHves ind comparsttv* forms 

d know* mooning and various veas of advarba ind comparative forms 

a *nows meaning and various uses of pronouns 

\ knows waning and various umi of conjunctions 

g know* meaning and various mi of prepositions 

h knowa meaning and venous uwi of interactions * 

i know* meaning and various uses of determiners (Roberts) v 

t know* meaning and vinous usss of "bo" (Roberta) 

k knowa moaning and various use* o- vorbaJs (Roberts) 

I knows mooning and various v*os of advortxais (Roberta) Onci oppositions! phrases) 

m knowa moaning and various uaoa of aantonco connocfors (Roborts) 



2 Sentence Construction 



s knows baste Engkeh aontonoo bottoms 

b knowa noon phraaos and verb phrases (Roborts) 



3 Sontorioo Transform stton ann £ pansWn 



a distinguishes •moos compound, complex, simple, end com pound -complex sentences 
b knows Roborts transformation rules X 



4 /oJntbot Usago 




a ensures agnaement Manse coo* andnumbj*' 
b onsuros agreomontwrih antocodonta 
c knows irreguiarittea 




i 




5 yorphotogy 




a undoratand* inflectional affixes 
b undofstand* oorivottooai affixes 








6 Phonology 




• understands varb forms 
b undorstands noun forms 
c undorstands pronoun forms 




\ 





t Prints nestiy accurately snd legibJy 



2 ftsdueot curstva writing lo no'rrsi aduM site 



3 Writes cursive with ei*e evenness speed 
accuracy naatneee sndteOibtMty 



4 Maintains good posture while writing 
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DIMOSI \ 

Directory of 'Sources of Measurable Objectives -First Editiok 



£veJ.oper ; 
Source : 



Cost: y 
6opyrig!it: 



Robert H. Pinney 

Upper 0 Midwest Regional Educational Laboratory 
1640 East 78th Street / 
Minneapolis, Minnesota 55423 



Contact source. 
Yes, 1972 



/ 



CONTENT AND ORGANIZATION: 

This model is, as its title indicates, a directory of sources of 
measurable objectives. Listed in this directory are what sets 
of objectives have been written, by whom, for what areas* of the 
curriculum, for what intended purposes, and in what format. , 

v 

This first .edition contains a listing of some 150 sets of objec- 
tives! from k variety of sources. In this manual, these listings 
are organized into 20 sections as follows: 



Agriculture 

Art ! 

Business \ 

Distributive Education 

English/Reading/Language 

Foreign Languages 

Health Occupations 

Health, Safety, Recreation, 

and Physical Education 
Home Economics 
Industrial Arts 



Mathematics 
Music 

Natural Sciences 

Office Occupations 

Social Sciences 

Technical Education 

Trade and Industrial 

General Elementary/Secondary 

Education 
Handicapped 1 ; 

Co-curricular Activities 



Soine categories have numerous listings. For several of the 
groupings listed, Aowever, no sets have yet^ been located. The 
information provided is brief and easy to discern, as it is 
organized in char^t form. 



SPLCJAL FEATURES: 



Any individual or group endeavoring to write sets of objectives 
or reviewing objectives banks would surely find this catalog of 
value. Its completeness is unique* 

4 ; 
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Instructional Objectives: A National Compendium 



ERIC 



Developers 



Source : 



Cost: 



Publication 
• Date : 

Copyright: 



Michael G. Kuhn and Lorraine R. Gay 

State University System of Florida Xallahassee 

Florida State Department of Education 

Division of Elementary and Secondary Education 

Tallahassee 

ERIC Docjament # f s ED 062 743 and EC 041 993 
c/c Computer Microfilm International Corporation 
Post Office Box 190 
Arlington, Virginia 22210 

Microfiche $0.65 
Hardcopy $6.58 



May 1972 
No 
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CONTENT AND ORGANIZATION ; 



This compendium is an annotated bibliography of materials con- 
taining instructional objectives and of objectives-based mate- 
rials collected from institutions, school systems, state 
departments of education, and commercial producers of educa- 
tional materials. Organization of listings is by topic, grade 
range, and alphabetical order. A listing of macro-collections, 
compilations which contain objectives in a number of different 
areas and many objectives per area, is followed by a listing 
of micro-collections, which contain objectives in at least three 
different areas. Other sections list materials containing edu- 
cational objectives afor specific curriculum areas: early child- 
hood, creative arts, language, mathematics, reading, science, 
social studies, and vocational-technical education. Final sec-' 
t^fbns list sources (bibliographies and reports) containing ob- 
jectives or references concerning objectives, materials dealing 
with development and use of objectives /' and teacher training 
materials which consist of or faontain instructional objectives. 
Information given, for each entry includes title, grade range, 

brief description of material, and a list of 
the material. ^ 



publisher, price, 
topics covered i|n 
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The Compendium is organized by topic, grade range and alphabet- 
ical order. Collections have been divided into MACRO-COLLECTIONS 
and MICRO-COLLECTIONS . MACRO- COLLECTIONS contain descriptions 
of compilations (arranged alphabetically) which contain objec- 
tives in a number of different areas and which contain many ob- 
jectives per area. MICRO-COLLECTIONS contain descriptions of 
collections which contain objectives in at least three different 
areas. These are arranged alphabetically within intended grade 
range. Descriptions of objectives in the curriculum areas, e.g. 
MATHEMATICS, .are also arranged alphabetically within grade range, 
with the exception of VOCATIONAL-TECHNICAL EDUCATION; descrip- 
tions in this category are ordered alphabetically by vocational 
area. SOURCES OF SOURCES contain descriptions of bibliographies 
and reports which contain either objectives or s references con- 
cerning objectives; these are arranged alphabetically by title. 
UTILIZATION OF OBJECTIVES contains descriptions of materials 
which deal with the development and utilization of instructional 
objectives, also arranged alphabetically by title. Finally, the 
TEACHER EDUCATION section contains annotations of available 
teacher training materials which consist of, or contain, instruc- 
tional objectives. These are arranged alphabetically, by topic, 
and by title within topic. SOURCES OF SOURCES, UTILIZATION OF 
OBJECTIVES, and TEACHER EDUCATION were net originally intended 
to be sections in the Compendium. However the number of unsolic- 
ited entries received warranted their inclusion. Consequently, 
in contrast to other sections of the Compendium, these three 
sections are in no way to be considered comprehensive. 

Descriptions of materials are presented in the words of the de- 
veloper whenever pc3sible. Annotations include the title of 
the materials, the intended grade range (K * kindergarten, P ■ 
primary, I = intermediate, JR = junior high, SH ■ senior high, 
A = adult, or post-secondary), name and location of the pub- 
lisher, and the price of the materials whejfi known. Since it j 
would be impossible to relate all important aspects of each re- / 
ported effort, this summary presents a bri'ef description of the J 
materials and a listing of the topics included. Using this ap- \ 
proach, interested persons may examine thjfe annotations in the J 
area or areas of their choice and identify sources of objectives 
relevant to their needs. ' \ \ 

To enhance the usability of this document, the name and addressj 
of the contact person for each entry in the Compendium are pre-, 
sented on the page opposite the page on which the entry origi- 
nates. Although many more persons were contacted, they either/ 
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did not respond, did not have materials to contribute, had ma- 
terials which are not yet available, or responded after the Com- 
pendium had been compiled. Late arrivals will be integrated 
into a revised version at a later date. 



5. PLAN; Master Objectives (K-SH) - Westinghouse Learning 
Corporation, Palo Alto, Calif ronia 

Description: A program of individualized education in 
Reading, Language Arts, ^Mathematics, Science, and Social 
Studies which can be adapted to each pupil's needs , abil- 
ities, interests' and goals. The major components of the 
program are a set of instructional objectives, a variety 
of study plans to achieve these objectives, and a com- 
puter support system to help plan student learning and 
monitor student progress. Objectives for the Guidance 
program are also included. 

Topics included: 



Primary Objectives 



Language Arts 



Science 



Orientation to PLAN 
Readiness 

Language Arts Skills 
Reading Skills 
Writing 



Biology 

Psychology 

Physics 

Chemistry 

Geology 

Meterology 

iAquiry Development - 
^Observing/Perceiving 
Inquiry Development - 



Social Studies 



History 



Sociology 
Geography 
feconomics 



Taxonomic/Classifying 
Inquiry Development - 



Experimenting 
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Mathematics 

Numerals * 

Place Value mo 9999 

Roman Numerals 

Addition of 4-digit Numbers 

Subtraction of 4-digit Numbers 

Multiplication Facts 
Division with 1-digit Numbers 
Addition of Like Fractions/ 
Patterns of Objects 
Functions of a Graph 

Geometric Figures 

Value of Coins 

Time to the- Minute 

Linear Measure to 1/4 inch 

Temperature Readings 

Elements of a Set 

Set Notation 

Number Sentences 

Word Problems 



Language Arts 

History & Dialectology 
Speaking 

Reading-Phonetic Analysis & Structural Analysis 
Reading-Critical Reading Skills 
Reading-Literature & Literary Form 
Writing-Correspondence Skills 
Listening 

Reading-Study Skills 
Reading-Organizational Skills 
Reading- Interpretation Skills 
Writing-Form & Analysis 
Writing-Creative Writing 
Writing-Structure 
Writing-Mechanics 
Writing-Study Skills 
Non-Verbal Communication 
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Developer: 

Rand Corporation 

Source: 

Rand Corporation 
.Santa Monica, Cal. 

Cost: 

$3.00 8 1/2" x 11" 
staple bound documents 

Copyright: 
1971, 1972 
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The MODIA reports are an attempt to apply systematic methods 
of ^analysis and synthesis to issues and problems in education, 
The MODIA reports present a systematic process for designing 
a mix of instructional facilities, materials, and students, 
and the process by which all of these elements might work to- 
gether to affect student mastery of subject matter. 



The MODIA systejra' consists of a sequence of procedures and semi- 
automated tools /that allow a designer to examine many alterna- 
tive structural/ approaches before a system is chosen* The 
utility of various approaches can be assessed in terms of stu- 
dent outcomes 1 or\ the consumption of human and material resources, 



The MODIA process consists of eight steps: s 

1. Analyze charac£eristics of the learner population that 
will affect the way the course will be taught, 

2. Use the Questionnaire for stating General' Policy to 
specify the broad goals underlying the teaching in- 
stitution 1 s operation. 

3a. Use the Curriculum Analysis Questionnaire to classify, 
each lesson in system-oriented terms, and to charac- 
terize each lessors requirements for communication 
media. 
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3b. Specify instructional strategies with the help of 
DISTAF (DETERMINING INSTRUCTIONAL STRATEGIES FOR 
TRAINING IN THE AIR FORCE) . Each strategy identifies 
the teaching agent and the way students will inter- 
act witlT this teaching mode, 

4. Specify design criteria input from the teaching in- 
stitution, such as least course cost, shortest course 
length, or maximum student graduation rate, 

5. Describe local resources, such as the number of stu- 
dents entering the school and the school's resources 
and constraints. 

6. Design the instruction using as direct inputs infor- 
mation from the Curriculum Analysis Questionnaire, 
DISTAF, the design criteria, and the local resource 
description. 

7. Analyse the costs to determine the system's time- 
dependent dollar requirements. 

8. As necessary, depending on the acceptability of the 
outputs, repeat steeps 1-7 (with different inputs) 
until the most desirable system emerges. 



The MODIA System was developed to meet specifications of the 
U.S. Air Force and consequently many of the descriptions don't 
fit the more general requirements of public education. As a 
result adaptations to local school situations will be needed. 
Also the consideration of highly varied strategies will ne- 
cessitate the utilization of a computer program available from 
the RAND corporation. The RAND corporation lists the follow- 
ing major tools that must be developed to complete the system: 
a generalized computer model of student flow, a computer model 
for estimating resource requirements and a questionnaire for 
detailing local resources. Obviously such sophistication in 
computer application in the most involved application would 
require outside consultation. 
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Although the decision process is intended to provide input for 
a semiautomated methodology for designing instructional pro- 
grams it also: 

1. assists the planner in making explicit judgments about 
what constitutes effective instruction. 

2. assists the planner in stating policies that guide 
1 school and classroom operations. 

3. provides a comprehensive, logically consistent set of 
decisions that should be considered in planning 
instruction, 

! 4. provides simple applications of the MOD I A system 

j which can be manually employed. 

! ' 

! 
: 

\ 

\ 

i 
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'■'Pathways to Success'' 



Developer; 

Dorothy Soper; N.J. Office 
of Program Development 



Source 5 

N.J. Div. of Research, 
Planning & Evaluation 
Project Center 
1000 Spruce Street 
Trenton, N.J. 08638 
(609) 292-8,454 

Cost: 

No charge for single 
copies in New Jersey 

Copyright : 
No (1974) 
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This manual was designed to contribute to the selection of 
successful educational programs developed and validated nation- 
ally by schools throughout the nation. The developers have 
also described it as a "development system" for education. 
When fully operative local educators will be able to identify 
local needs for development work, use basic research to design 
and field test new programs, and make successful programs 
available to those wishing them. 



1) Identification of an educational need for which no program 
is available in the district; 2) Define results desired; 
3) Review existing programs for their applicability; 4) Secure 
pertinent available materials; 5) Replicate a program developed 
in other districts for which materials and training are avail- 
able; 6) Replicate a program developed by other agencies for 
which materials and training are available; 7) Consult appro- 
priate research; 8) Design, field test, and evaluate a new 
program; 9) If program is successful, make it available to 
educators beyond originating district. 
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The cycle which produced the innovation is the "how to do it" 
information that will permit a "consumer" district not only 
to use a new program or materials, but also to adapt the in- 
novation to the unique requirements of a different situation 
through an application of the problem-solving process itself. 
Also, the program must be based on research findings, have 
specific measureable goals, a precise plan of action, and an 
evaluation design. The program must have been successful in 
having met its predetermined goals. A final concern is that 
the program be cost effective for the consumer district ♦ In 
other words the costs required to maintain the program and 
improve performance should not generally exceed existing ex- 
penditures. 



The consumer district should evaluate the suitability of a 
program in comparison to its own educational needs, goals, 
population and resources. This evaluation should indicate 
appropriateness and feasibility. Also, an adopting district 
must indicate willingness to submit to evaluation and provide 
the costs involved in providing time for travel, training, 
and instructional materials. 



The model is a. turnkey approach to program implementation. 
Training in the problem-solving process as related to future 
educational needs is included . 
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This model introduces the multidimensional efforts involved in 
conducting a full program evaluation. The focus is paroramic 
rather than microscopic. The* broad intent of the model is to 
discover congruences or contingencies between unstructural in- 
tents and student outcomes. A series of matrices are proposed 
for the collection and judgment of data. 



Evaluators work from a series of data matrices. The procedure 
is to collect and/or prepare such information as is needed 
under three general headings, i.e., 1) a "description- judgment" 
matrix including descriptions of instructional intents, inter- 
actions or transactions, and outcomes; 2) a matrix for proces- 
sing descriptive data; and 3) a matrix for making comparisons 
between programs and standards of excellence, and their result- 
ant judgments. The evaluator's first responsibility is to de- 
scribe in adequate detail the antecedents (baseline and contribu- 
tory factors) , transactions (teacher-pupil/teacher administrator 
etc. interactions) , and outcomes (both learner and staff) for 
the project in question. These three levels of data are then 
placed on a four heading format such that each level of input can 
be compared (evaluated) relative to intents, observations, stan- 
dards and judgments. In other words data from the descriptive 
matrix is juxtaposed with data from a judgment matrix. The 
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model's author contends that it is th^^Valuator 1 s responsibil- 
ity to prepare such objectives as are to be included in the 
evaluation. Having prepared the descriptive data necessary for 
intended antecedents, transactions and outcomes, he then enters 
the corresponding data for observed antecedents, transactions 
and outcomes. Judgments of congruence are then made. This mod- 
el also provides for judgments on the logical and empirical con- 
tingencies between descriptive and observed data. The final 
procedure is that of making relative comparisons between: 

• Descriptive data from similar programs 

• Descriptive data from the project in question 

• Standards of excellence/resulting judgments 

Also, the model provides distinctives between relative and ab- 
solute judgments, and suggests elements for the evaluation de- 
sign's rationale. 

/ 

This model is one of the pioneers in the formulation of new 
approaches to evaluation. While the material is somewhat 
dated, there are still many strengths to the comparative pro- 
cedures proposed. The primary usefulness of the model lies in 
the contribution it makes, to the construction of a rationale. 
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This booklet has been written to assist the reader in the fol- 
lowing areas: 

1. Defining criterion-referenced tests 

2. Reviewing the advantages and disadvantages of criterion- 
referenced tests 

3. Determining how norm-referenced and criterion tests differ 

4. Utilising criterion-referenced results 

* 

# 

The booklet is written in response to six questions: 

1. What is a criterion-referenced test? 

2. How do criterion- and norm- referenced tests differ? 

3. What are the advantages and disadvantages of a criterion- 
referenced test? 

4. How are criterion-referenced tests constructed? 
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5. .How are reliability and validity established for criterion- 

referenced tests? 

6. How are criterion-referenced test results reported? 

Additionally , EPIC has available criterion-referenced tests that 
incorporate seven sequential program studies-lists of the basic 
skills and concepts included in the specific curriculum areas to 
be assessed. Each skill and concept is tested by four different 
test items of equal difficulty. These tests are performance ob- 
jective based, and geared to learners with different test-t,aking 
abilities. Standardized test procedures advance in sophistication 
as the student advances in test- taking ability. The tests are 
designed to: 



| • determine what sk 



ills and concepts the learner has mastered 
| * identify needs related to these skills and concepts 

■ assess performance objectives ^ 

j # comply with federal program evaluation guidelines 

i * 

! In constructing test -items the school district may select items 
, from EPIC's item pool or c choose to .develop its own items with 
technical assistance from EPIC consultants. 



ERLC 



201 or>.o 



£02 



Developers: 

Wells Hiyely, Graham 

Maxwell, 'George Rabehl , 

Donald Sension, Stephen 

Lundin 

Source s 

Center for thp Study of 

Evaluation 

U.C.L.A. I 

Los Angeles, Cal. 90024 

Cost: 

Contact Source 

Copyright: 
1972 



/ 71 

/ COMPONENTS IN MODEL f\ 




\ 

This\ monograph has „three interwoven themes. It contains 1) a 
technical handbook for the application of doamin- referenced 
test theory to educational evaluation, 2) a case history of the 
curriculum project in which the technology was developed, and 
3) an essay on strategies of educational development and evalu- 
ation* By interweaving the three theories, the desire is to 
illuminate some of the technical and strategic problems of edu- 
cational development from angles that are hard to represent in 
shorter theoretical articles or technical reports. 



The exposition is as follows: Chapter I gives a brief summary 
of the history and mode of operation of the Minnesota Mathe- 
matics and Science Teaching Project (MINNEMAST) . Chapter II 
treats the rationale that underlines domain- referenced achieve- 
ment testing and the technology through which it was applied 
to evaluation of the MINNEMAST Curriculum. Chapter III ex- 
amines organizational factors that have in the past limited the 
effectiveness of empirical, formative evaluation in many large- 
scale curriculum projects and suggests an alternative strategy 
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for the future. Readers mainly interested in evaluation tech- 
nology should concentrate bn Chapter II. Those whose main 
interest is in strategies of educational development and evalu- 
ation should concentrate on Chapter III, skrmming Chapters I 

/ and II to put meat on the skeleton of generalities encountered 
there. Those whose main interest :Ls in the history of the 
evaluative effort in the MINNEMAST Project should read Chapters 
I and, III, skimming Chapter II. To escape from superficiality, 
an adequate exposition of domain-referenced curriculum /evalu- 
ation' required a large fund of examples. These have b£en col- 
lected into several appendices. Two particularly important 
cases dan only be made by examining the appendices and 1 are 
worth emphasizing here. First, that the "item form tr Approach 
is not simply a way to operationally define pre-specif;ied goals 

) but is much more fundamentally a way of teasing out and dis- 
covering implicit goals. Second, that the power of the "item 

*s form" approach lies in thc^ discovery of patterns of student 
performance that pinpoint the dimensions along which success 
(or failure) generalizes. 
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OVERVIEW: , \ 1 

This book is one, of the prpducts of the Phi Delta Kappan (PDK) 
National Study Committee oh Evaluation which was established 
for the purposes of: 1) delineating the status of our knowledge 
about the process of evaluation; 2) collecting, analyzing, and 
providing critiques of models of evaluation that had been pro-= 
posed in the past; 3) synthesizing the strengths of existing 
models into one which had the potential for meeting the demands 
of the times; and 4) describing' that model in language meaning- 
ful to the practitioner assigned program evaluation responsi- 
bilities. 
* 

The general purposes of, this book are to deal with the kinds of 
problems evaluation raises and to devise appropriate solutions. 
There are four specific objectives: 

1) To expose, in detail, five problem areas: definition, de- 
cision making, values and criteria, administrative levels, 
and the research model. 

2) To identify and assess ^extent or emergent formulations th^t 
° might be used ±tv conceptualizing solutions to these five \ 

problems, and to fashion them into tentative solutions. 

3) To synthesize a new de^ initton^and methodology of evaluation 
that builds upon the products of^bbjeLCtive 2. 
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4) To provide some operational guidelines for implementing the 
proposed new approach in terms of personnel, organization, 
and administration. j 

This is a book for a variety of audiences directly or indirectly 
concerned with the art and science of evaluation, including not 
ofily the producers and consumers of evaluation but the theoreti- 
cians, teachers, consultants, measurements experts, systems an- 
alysts, statisticians, data processors, computer specialists, 
and others who must relate to it in one way or another. 

/ 

Issues considered included i decision-making behavior of admin- 
istrators, theoretical formulations of evaluation and decision* 
making, value conflicts, multiple information requirements at 
different organizational levels, appropriateness of extant re- 
search techniques for use in evaluation, the role of k evaluation 
systems, and the training of evaluators. These disparate is- 
sues were considered appropriate for consideration in this book, 
since a comprehensive solution to the many problems which block 
spund evaluation was being sought. 

The analysis and synthesis contained in this book reveal that 
* no single group possesses all of the competencies necessary /to 
provide effective and lefficient evaluation. Nothing short of 
a concerted team effort will do. Administrators, curriculum 
developers, evaluators,, research methodologists , communications 
experts, theoreticians, philosophers, planners, and educational 
technologists, among others, must contribute from their special 
areas of expertise to the solution of evaluation problems. * 

To be adequate s to the overall task, these contributions must be 
coordinated. Such coordination can be effected through tthe es- 
tablishment of special evaluation units in educational institu- 
tions. Thus, this book is directed at those who commission the 
development of /evaluation units, operate the units, use the in- 
formation produced by^ the units, comprise evaluation teams, pro- 
duce methods and materials for use in evaluation units, and 
train staff fh* work in evaluation units. 



PROCEDURES : 

The basic procedures of a sound model for educational evaluation 
are distributed under four major headings and sixteen steps as 
follows : 
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Planning Decisions — General Setting 

Step I - Awareness (Monitor program to identify needs and 
opportunities and rank needs and opportunities.) 

Step II - Design (Identify alternative conceptions of the 
probleija to be solved and specify the problem selection 
parameters. ) 

Step III - Choice (Assess alternative probl em st atements 
from different value positions and select the problem.) 

Step IV - Action (Assess whether the' problem solution witlT 
require large or small change and the degree to jwhicH in- 
formation is available to guide the change arcftivity, and 
specify ?the objectives.) 

Structuring Decisions — Neomobilistic Setting 

Step V - Awareness (Enlist the services of the evaluator 
and identify the structuring questions that follow from 
the change objective.) 

Step VI - Design (Choose a decision model, set problem 
parameters and devise several solution strategies.) 

<? 

Step VII - Choice (Assess the alternative strategies and 
choose the best alternative.) 

Step VIII - Action (Perform structured inquiry to assist 
in programming the chosen change strategy and program the 
strategy over a long time span.) 



Implementing Decisions 

Step IX - Awareness (Identify and monitor barriers to suc- 
cess- and determine when a barrier should be ameliorated.) 

Step X - Design (Continue at leas£ one design for modify- 
ing the design or procedure.) 
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Step XI - Choice (Assess proposed change and decide whether 
or ribt to use t one of the proposed changes.) 

Step XII - Action (Adjust the evaluation design and program 
changes decided upon.) 



4. Recycling Decisions 
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Step XIII - Awareness (Monitor attainments to identify dis- 
crepancies between performance and objectives , and determine 
whether to attend to a discrepancy.) 

\ ! 

Step XIV - Design (Conduct structured inquiry and conceptu- j 
alization to identify possible causes of the discrepancy , 
determine probable causes , and formulate responses.) 

Step XV - Choice (Conduct means-ends analysis, if required> 
and decide whether to recycle the change procedure.) j 

Step XVI - Action (Prescribe what changes , if any, are to 

be effected.) ! 



The purpose of the evaluation unit should be described in 
detail through a specific set of objectives. 

The evaluation unit should be staffed in accordance with 
the many roles to be performed in evaluation. 

Agency policies must be updated, to include policy state- 
ments governing access to date, dissemination of reports 
and information, and activities involving unit personnel 
in their daily operations. j 



Adequate facilities and equipment Should be provided for 
evaluation. Another important characteristic of evalua- 
tion units is the equipment and facilities required to 
implement programs for evaluation^ 

Underfinanced programs providing limited personnel and 
facilities can only lead to inadequate information for 
decision making. ! 
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6. Eliminate the anxiety symptom by creating a healthy environ- 
ment for evaluation. 



SPECIAL FEATURES: 

The book is designed according to the format of the model of 
evaluation it extends. The content is comprehensive and ad- 
dresses the practical application of the evaluation modfel. 
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OVERVIEW: 

< 

This model is a workshop study guide which is appropriate for 
teachers, supervisors , department heads, and principals who 
are concerned with the translation of educational goals into 
precise program objectives that can^ then be measured and op- 
erationalized. Emphasis is placed on three phases of program 
planning: 1) preliminary stages (planning and assessment) , 
2) intermediate stage (programming, evaluation, test selection, 
data collection, and objective refinement), and 3) final stage 
(mechanics of model development, administrative operations and 
techniques dealing with H How to do it, H and, "How much it costs.' 

This model presents the concept of a program development system 
in terms that are relevant to classroom teachers. Workshop 
participants are led through each stage of model development 
with refinement exercises provided in the text materials to 
translate needs to goals and goals to objectives which are de- 
signed to embody measurable performance standards. 

•IMPLEMENT AT ION NEEDS : 

The model should be implemented through some form of trained 
leadership to direct and coordinate the workbook exercises. A 
serious study group, however, could work through the model us- 
ing only the publication. The self testing exercises throughout 



ERLC 



209 



210 



